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This  report  is  essentially  a  transcription  of  a  presentation  given  at 


the  Resource  Analysis  and  Management  Working  Group  of  the  48th  Military 
Operations  Research  Symposium.  Because  the  topic  discussed,  the  vali¬ 
dation  of  cost  allocation  methodologies,  may  be  of  interest  to  a  wider 
audience,  we  have  prepared  this  report  for  dissemination. 

Slide  #1 

In  this  paper  we  present  a  general  discussion  of  a  difficult  vali¬ 
dation  problem — the  validation  of  the  algorithms  used  to  allocate  oper¬ 
ating  and  support  (O&S)  costs  in  a  military  cost  reporting  system.  It  is 
based  on  a  study  currently  being  conducted  by  Desmatics,  Inc.  for  the  Office 
of  VAMOSC  II,  within  the  Air  Force  Logistics  Command  (AFLC) ,  USAF. 

Cost  allocation  is  an  imprecise  art  at  best;  cost  accountants  have 
wrestled  for  years  with  the  problems  involved  in  allocating  factory  over¬ 
head  among  production  departments.  The  military  counterpart  to  that  problem 
is  much  more  complex  than  that  usually  encountered  in  the  typical  industrial 
model . 

S 1  ide  ft 2 

Although  this  paper  is  intended  to  have  broad  applicability  in  the  area 
of  cost  allocation  validation,  it  will  be  discussed  in  terms  of  a  specific 
system  currently  under  development  by  the  U.S.  Air  Force.  This  system  is 
called  the  Weapon  Systems  Support  Cost  (WSSC)  system.  It  is  a  subsystem 
of  the  Visibility  and  Management  of  Support  Costs  (VAMOSC)  system.  Actually 
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the  Air  Force  refers  to  its  VAMOSC  system  as  VAMOSC  II,  since  the  system 
now  under  development  is  a  replacement  for,  and  considerable  enhancement  of, 
an  existing  system  which  has  been  in  operation  since  1976.  WSSC  is  concerned 
with  aircraft  weapon  systems  as  distinguished  from  large  ground-based  radar/ 
communication  systems  or  system  "black  box"  components.  These  two  categories 
are  handled  by  the  Communication-Electronics  System  (C-E)  and  the  Component 
Support  Cost  System  (CSCS) . 

It  should  be  pointed  out  that  the  WSSC  system  deals  with  actual,  incurred 
costs  for  the  operation  and  support  of  aircraft  weapon  systems.  The  costs 
portrayed  in  WSSC  do  not  include  any  R&D  or  production  costs.  Furthermore, 

WSSC  is  a  cost  collection  system,  not  an  accounting  system  or  a  cost  estimat¬ 
ing  system. 

Slide  # 3 

WSSC's  primary  product  is  an  annual  report  which  provides  a  cost  break¬ 
down  for  each  fleet  of  aircraft  weapon  systems,  identified  by  MDS  (mission- 
design-series),  such  as  B52H,  C5A,  F15A.  Slide  3  illustrates  in  highly  sim¬ 
plified  format  the  type  of  operating  and  support  cost  data  presented  in  the 
primary  WSSC  system  report  product,  the  WSSC  AF  Detail  Report.  Shown  here  are 
just  four  of  the  major  cost  categories  into  which  O&S  costs  are  divided.  Each 
of  these  is  further  broken  down  into  two  or  more  subcategories.  For  example, 
unit  operations  include  aircrew,  command  staff,  security,  fuel,  munitions 
training  and  other  unit  costs.  Installation  Support  includes  real  property 
maintenance  (RPM) ,  base  operating  support  (BOS)  and  communications  (COM). 

A  factor  common  to  most  of  the  cost  categories  is  that  the  cost  information 
required  to  fill  in  this  matrix  is  usually  not  available  directly,  but  must  be 
allocated  to  the  MDS  level.  Within  these  cost  categories  are  several  unique 
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allocation  problems.  For  instance  fuel  cost,  medical  cost,  aircrew  cost* 
maintenance  cost  and  installation  support  cost  data  are  each  available  from 
different  sources  and  at  different  levels  of  identification,  thus  necessitat¬ 


ing  different  treatments  to  get  the  required  costs  at  the  MDS  level.  This 
means  that  several  different  allocation  algorithms  must  be  used  by  WSSC . 

Slide  #4 

This  slide  illustrates  the  flow  of  data  into  the  WSSC  system.  Most  of 
the  O&S  costs  are  incurred  at  airbases  within  the  several  commands  of  the 
U.S.  Air  Force.  Data  flows  up  through  commands  into  several  data  systems 
which  existed  before  VAMOSC  and  WSSC  were  designed. 

An  important  constraint  on  WSSC  operation  is  that  WSSC  must  utilize  data 
from  existing  systems;  separate  data  collection  facilities  may  not  be  con¬ 
structed  for  WSSC.  Much  of  the  cost  information  required  by  WSSC  is  avail¬ 
able  through  the  Air  Force!s  Accounting  and  Budget  Distribution  System  (ABDS) . 
However,  fuel  costs  and  depot  maintenance  costs  must  be  obtained  from  other 
sources,  as  shown  in  the  illustration. 

The  cost  data  in  ABDS  is  reported  by  base  and  is  identified  by  program 
element  codes  (PEC) ,  responsibility  center/cost  center  codes  (RC/CC)  and 
element  of  expense/investiment  codes  (EEIC) ,  but  not  by  MDS.  Thus,  to  get 
the  required  fleet  costs  for  each  MDS,  it  is  necessary  to  classify  costs  by 
type,  apply  appropriated  allocation  algorithms  at  base  level  and  roll  these 
costs  up  to  fleet  level. 

Slide  //5 

Personnel  strength,  flying  operations  and  maintenance  manhour  data  are 
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utilized  by  WSSC  as  the  basis  for  allocating  costs  to  aircraft.  Slide  5 
summarizes  the  types  of  data  used  by  WSSC  for  cost  allocations.  Personnel 
strengths  come  from  the  E300Z  Advanced  Personnel  System.  Flying  hours  and 
possessed  hours  (from  which  aircraft  counts  are  derived)  come  from  the  G033B 
AVISURS  system.  Maintenance  manhours  are  obtained  from  the  E506  AMMIS  System. 

In  this  paper  we  will  not  describe  all  of  the  categories  used  in  WSSC, 
nor  will  we  discuss  all  the  allocation  algorithms  employed  by  WSSC.  We  will 

concentrate  instead  on  just  one  of  these  cost  categories - installation 

support  costs - since  they  resemble  the  typical  overhead  allocation  problem 

encountered  in  industrial  cost  accounting. 

Slide  //6 

This  slide  shows  the  task  which  WSSC  must  perform  with  respect  to  the 
installation  support  costs  incurred  at  base  level.  ABDS  reports  the  annual 
costs  by  base  as  a  number  of  individual  line  items.  Some  of  these  items  are 
clearly  fixed  costs  which,  according  to  DOD  guidelines,  are  not  to  be  allocated 
to  aircraft.  Other  costs,  such  as  BOS,  RPM,  and  COM,  have  both  fixed  e  l  vari¬ 
able  components.  The  fixed  components  cannot  be  identified  directly,  so  a 
portion  should  be  factored  out  of  the  total.  The  variable  costs  (the  portion 
remaining  after  the  fixed  component  has  been  removed)  constitutes  the  total 
burden  to  be  allocated  among  all  tenants,  including  both  the  aircraft  weapon 
systems  at  that  base  and  any  non-aircraft  tenants.  This  requires  application 
of  another  allocation  algorithm. 

The  remaining  BOS,  RPM  and  COM  costs  constitute  the  aircraft  variable 
cost  burden  which  is  to  be  allocated  among  the  base  aircraft.  The  question 
is,  "What  is  the  most  reasonable  basis  for  allocating  these  costs?".  In  this 
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example  there  are  72  TAC  fighters,  15  SAC  bombers  and  15  SAC  tankers  at  the 
same  base,  closely  resembling  the  configuration  on  at  least  one  AF  CONUS 
base.  The  flying  hours,  crew  sizes  and  total  supported  strengths  represent 
typical  figures  taken  from  an  Air  Force  planning  document*.  They  are  repre¬ 
sentative  of  the  type  of  information  available  for  use  in  allocation  of  over¬ 
head  costs.  In  this  example  the  fighters  have  more  flying  hours,  crew  and 
total  strenth  and  probably  should  bear  the  majority  of  the  overhead  burden. 
Between  the  bombers  and  tankers,  however,  the  picture  is  mixed.  The  tankers 
fly  more,  but  the  bombers  require  more  personnel  for  whom  support  costs  must 
be  incurred. 

That  essentially  is  the  type  of  allocation  problem  which  a  system  such 
as  the  WSSC  system  must  address  in  order  to  provide  the  type  of  cost  break¬ 
downs  required  to  achieve  the  necessary  visibility  of  operating  and  support 
costs  for  aircraft  weapon  systems.  The  main  features  of  the  problem  are: 

(1)  a  requirement  to  provide  costs  at  the  weapon  system  level,  (2)  avail- 
ablility  of  cost  data  from  several  sources,  but  lack  of  available  data  at 
the  required  level,  thus  necessitating  the  use  of  allocation  algorithms,  and 
(3)  availability  of  some  (possibly  conflicting)  types  of  data  for  use  as 
the  basis  of  allocation. 

The  task  in  validating  the  specific  choices  made  in  the  design  of  a 
military  cost  allocation  system  is  to  identify  the  reasonable  alternatives 
and  assess  them  in  a  framework  which  is  as  objective  as  the  situation  permits. 
The  remainder  of  this  paper  addresses  a  general  methodology  for  completing 
this  task,  using  WSSC  as  an  illustrative  case. 

*AFP  173-13  "USAF  Cost  and  Planning  Factors,"  1  February  1980. 
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Slide  #7 


In  a  general  context,  there  are  a  number  of  cost  categories  (N)  to  be 
considered.  For  each  cost  category,  we  have  the  total  expenditures  over  all 
MDS's.  These  total  expenditures  must  be  apportioned  to  each  MDS,  often  by 
means  of  an  allocation  algorithm. 


Slide  # 8 

The  purpose  of  the  type  of  validation  effort  discussed  in  this  paper 
is  to  examine  critically  the  underlying  algorithms  for  allocating  the  costs 
to  each  MDS.  It  is  not  concerned  with  the  related  task  of  determining  whether 
the  WSSC  computer  programs  are  correct,  i.e. ,  whether  the  algorithms  have  been 
coded  correctly  and  thoroughly  debugged. 

If  we  consider  ,  the  costs  in  category  j  allocated  by  WSSC  to  MDS  i, 

the  validation  task  we  are  discussing  has  as  its  objective  the  comparison  of 

with  T^  ,  the  corresponding  "true"  costs.  Of  course,  because  we  are  con¬ 
cerned  with  each  of  a  number  (K)  of  MDS’s  and  each  of  N  cost  categories,  we  can 
consider  the  overall  task  as  comparing  the  NxK  matrix  with  the  NxK  matrix  T\ 

Notice,  however,  that  the  term  "true"  costs  was  put  in  quotations.  This 
indicates  that,  in  many  cases,  the  actual  costs  that  should  be  assigned  to  a 
given  MDS  are  unknown,  except  perhaps  by  an  omniscient  being. 


Slide  j 9 

Thus,  direct  validation  by  examination  of  the  differences  between  the 
WSSC  allocation  cost  matrix  C  and  the  true  cost  matrix  T  is  not  possible,  in 
general.  Instead,  any  validation  effort  must  take  an  indirect  route.  Such 
indirect  validation  is  concerned  with  two  general  types  of  validity.  One, 
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face  validity,  involves  a  qualitative  assessment,  while  the  other,  mathe¬ 
matical  validity,  involves  a  quantitative  assessment. 

Slide  //10 

Face  validity,  as  we  will  use  the  term,  refers  to  the  examination  of  the 
algorithms  on  a  subjective,-  common-sense  basis.  In  examining  face  validity, 
we  must  ask  whether  the  allocation  algorithms  appear  reasonable,  particularly 
when  compared  to  possible  alternative  allocation  schemes.  We  must  also  con¬ 
sider  whether  the  algorithms  appear  to  provide  equitable  results,  e.g.,  whether 
the  choice  of  an  allocation  basis  is  fair,  and  does  not  make  one  group  of  MDS's 
bear  an  inordinate  amount  of  the  costs. 

We  must  also  ask  ourselves  if  the  data  base  can  be  improved.  Perhaps  the 
data  base  currently  used  in  cost  allocation  can  be  replaced  by  a  different  data 
base  which  permits  allocation  at  a  lower,  and  thus  more  accurate,  level.  Al¬ 
though  the  data  base  really  does  not  form  a  part  of  any  allocation  algorithm, 
a  good  algorithm  can,  of  course,  be  made  useless  if  it  is  based  on  inaccurate 
data.  Thus,  some  assessment  must  be  made  of  data  base  accuracy. 

In  conjunction  with  each  of  these  steps,  a  major  check  on  face  validity 
is  to  consult  with  potential  users  throughout  the  development  of  a  system 
of  cost  allocation  algorithms.  Their  suggestions  and/or  criticisms  can  often 
result  in  major  improvements  which  yield  increased  face  validity.  As  an  aside, 
we  should  note  that  the  Air  Force  Office  of  VAMOSC  has  worked  closely  with 
potential  users  of  WSSC  during  its  development. 

Slide  HI 

Parallel  with  the  consideration  of  face  validity,  we  must  also  look  at  the 
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algorithms  from  a  quantitative  standpoint.  This  involves  the  concept  of 
mathematical  validity,  which  refers  to  the  quantitative  evaluation  of  the 
mathematical  framework  underlying  the  cost  allocation  algorithms.  We  must, 
as  part  of  this  type  of  validation,  examine  whether  the  algorithms  are  con¬ 
sistent. 

In  general,  an  allocation  algorithm  involves  (implicitly  or  explicitly), 
one  or  more  parameters  to  which  values  must  be  assigned.  A  major  question, 
therefore,  is  how  sensitive  the  cost  allocations  are  to  the  assigned  para¬ 
meter  values.  If,  for  example,  we  find  that  cost  allocations  are  very  sen¬ 
sitive  to  a  certain  parameter,  more  study  could  be  focused  on  that  parameter 
and  the  associated  algorithm. 

We  should  also  examine  whether  the  parameter  values  used  in  the  algorithms 
appear  reasonable.  For  example,  in  a  particular  algorithm  involving  a  para¬ 
meter  r,  does  a  value  of  r=100  appear  more  reasonable  from  a  mathematical 
viewpoint  than  does  a  value  of  r=5?  Also  we  should  ask  whether  the  para¬ 
meter  values  are  stable.  That  is,  can  the  same  value  of  r  be  used  year  after 
year,  or  must  it  be  revised  each  year? 

Slide  // 12 

Slide  //12  presents  an  example  of  a  consistency  check.  Let  us  assume 
that  two  cost  allocation  algorithms  are  being  examined,  and  that  each  algorithm 
uses  two  items  of  data,  and  X2  •  as  the  basis  for  allocation. 

Consider  situation  //l.  Here,  each  algorithm  provides  the  same  allocation 
of  costs  to  the  four  MDS's.  Now,  let  us  look  at  situation  #2,  where  the  data 
being  used  for  allocation  (X^  and  X^)  remains  identical  to  that  in  situation 
//I  except  for  MDS  //3  for  which  the  values  of  both  X  and  X9  have  increased. 


A 
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One  of  the  first  things  we  see  in  this  situation  is  that  both  of  the 
algorithms  result  in  the  same  allocation  of  costs  ($300K)  to  MDS  # 1.  We 
might,  therefore,  venture  that  both  algorithms  are  inconsistent  because 
they  do  not  allocate  $240K  as  in  situation  #1.  We  might  reason  that  since 
X^=  200  and  X2=  1000  for  MDS  #1  in  both  situations,  the  identical  cost 
amount  should  be  allocated  to  this  MDS  in  both  situations.  However,  such 
a  conclusion  would  be  incorrect  because  other  variables  do  not  remain  the 
same  from  the  first  to  the  second  situation.  At  the  very  least,  the  sit¬ 
uations  must  be  separated  in  time.  Thus,  the  assumption  of  "ceteris  paribus" 
cannot  be  made.  Hence,  the  allocation  of  $300K  instead  of  $240K  is  not  an 
indication  of  inconsistency. 

However,  since  and  X^  are  the  basis  for  allocation,  the  relative 

costs  allocated  to  MDS  #1,  MDS  #2  and  //4  should  be  the  same  in  both  situations. 
Thus,  since 

240:420:460 

are  in  the  same  ratio  as 

300:525:573  , 

allocation  algorithm  //I  is  consistent  in  this  respect.  However,  algorithm  #2 
is  not  consistent  because 

240:420:460 
are  not  in  the  same  ratio  as 
300:480:500  . 


Slide 

As  far  as  sensitivity  is  concerned,  we  can  examine  the  relative  sensitivity 
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within  a  cost  category  (i.e.,  the  proportional  change)  based  only  on  the 
allocation  data.  However,  to  get  an  idea  of  absolute  sensitivity  (i.e.,  the 
actual  change  in  dollars),  we  would  need  the  total  dollar  amounts  included 
in  each  category. 

In  any  event,  to  evaluate  the  sensitivity  of  cost  allocation  algorithms 
to  a  particular  parameter  r,  we  can  examine  the  partial  derivative,  taken 
with  respect  to  r,  of  the  relative  (or  absolute)  allocated  costs  or  we  can 
investigate  the  extremes,  i.e.,  the  allocated  costs  if  r  is  at  its  lowest 
possible  value  compared  with  the  allocated  costs  if  r  is  at  its  highest 
possible  value. 

Slide  £14 

In  this  slide,  we  present  a  hypothetical  situation  involving  what  is 
essentially  one  of  the  WSSC  cost  allocation  procedures,  the  use  of  a  weighted 
average  of  flying  hours  (FH)  and  possessed  hours  (PH) .  This  weighted  average 
is  of  the  form 

pFH  +  (l-p)PH 

where  0<p<l.  This  weighted  average  involves  one  parameter,  p,  which  can 
range  in  value  from  p=0  (allocation  based  completely  on  possessed  hours) 
to  p=l  (allocation  based  completely  on  flying  hours) , 

The  simple  illustration  presented  here  provides  an  example  involving 
only  four  MDS's.  (WSSC  deals  with  approximately  one  hundred  MDS's.)  Here 
we  have  considered  sensitivity  of  the  cost  allocations  based  on  the  minimum 
and  maximum  values  of  p.  As  can  be  seen,  MDS  // 3  is  most  sensitive  as  p 
varies  from  0  to  1 ,  with  the  corresponding  change  of  -23%  on  a  relative 
scale  or  -$460K  on  an  absolute  scale  assuming  a  total  of  $2000K  to  allocate. 
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If  we  examine  the  distribution  of  the  sensitivity  values  by  plotting 
them,  we  may  gain  a  great  deal  of  information.  Of  course,  in  our  simple 
example  of  four  MDS’s,  plotting  is  not  worthwhile  as  it  would  be  when  used 
with  actual  WSSC  data. 

Slide  //15 

It  should  be  noted  that  because  of  the  way  the  sensitivity  values  are 
defined,  their  mean  is  always  zero.  Therefore,  we  know  that  to  some  degree 
the  sensitivity  values  will  cluster  around  zero.  What  we  need  to  examine  is 
how  they  cluster.  Specifically,  we  must  pay  attention  to  the  spread  in  the 
data,  to  the  existence  of  outliers,  and  to  the  existence  of  any  particular 
patterns . 

For  example,  consider  the  three  sensitivity  value  distributions  in  the 
lower  half  of  the  slide.  We  certainly  would  have  more  concern  about  the 
second  and  the  third  than  we  would  about  the  first.  Our  concern  about  the 
second  distribution  would  be  focused  on  its  wide  spread  which  tells  us  that 
the  cost  allocations  are  very  sensitive  to  the  value  selected  for  the  para¬ 
meter.  Thus,  particular  attention  should  be  paid  to  the  specification  of 
this  value.  For  the  third  distribution,  there  are  indications  that  a  few 
MDS's  are  extremely  sensitive  to  the  assignment  of  the  parameter  value.  In 
this  case,  attention  must  be  paid  to  these  particular  MDS*s  to  determine 
whether  perhaps  a  revised  allocation  algorithm  might  be  required. 

Slide  1 6 

As  previously  noted,  we  cannot  directly  verify  the  costs  allocated  to 
each  MDS.  However,  if  data  from  a  number  of  years  is  available,  we  can  ex- 
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amine  stability /reasonableness  by  looking  at  the  relationship  between  total  j 

costs  for  a  given  cost  category  and  the  total  allocation  basis,  which  in-  j 

i 

volves  a  corresponding  parameter  vector  _r.  In  other  words,  for  a  given  j 

cost  category  such  as  installation  support,  we  can  check  the  relationship 
between  total  costs  and  the  total  allocation  basis  when  different  allocation 

I 

parameters  are  used,  \ 

In  general,  we  would  hope  that  for  the  best  choice  of  the  parameter 
vector  _r  there  would  be  a  relatively  strong  relationship,  at  least  when 
total  costs  are  adjusted  for  the  effects  of  other  variables  (for  example, 
by  means  of  the  analysis  of  covariance) .  In  exploring  such  adjustments, 
examination  of  existing  cost  estimating  relationships  may  prove  of  value. 

If,  for  a  given  value  of  the  parameter  vector  £,  we  obtain  a  good 
relationship,  it  would  probably  be  best  to  use  that  value  of  _r  in  the 
allocations,  assuming  that  there  is  no  evidence  to  the  contrary.  For 
example,  in  this  slide,  the  use  of  the  parameter  vector  value  r^  provides 
a  better  relationship  than  does  .  Thus,  _r^  is  probably  a  better  value  to 
use.  Of  course,  if  we  obtain  negative  results,  this  would  reflect  on 
either  unreasonableness  or  on  lack  of  stability. 

Summary 

In  summary  we  must  point  out  that,  to  many  people,  the  term  validation 
has  the  connotation  of  proving  that  a  cost  allocation  methodology  is  either 
correct  or  incorrect.  Actually,  the  most  a  validation  effort  can  usually  do 
is  to  provide  increased  confidence  in  the  procedure  if  no  defects  are  re¬ 
vealed.  We  hope  that  this  paper  has  provided  some  general  guidelines  that  may 
prove  of  value  in  validation  studies  concerned  with  systems  such  as  WSSC.  j 
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